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ABSTRACT 


This  final  report  describes  the  research  carried  out  by  members 
of  the  Decision  and  Control  Sciences  Group  at  the  Laboratory  for 
Information  and  Decision  Systems,  M.l.T.  during  the  time  period 
February  1,  1977  to  January  31,  1982  with  support  extended  by  the 
Air  Force  Office  of  Scientific  Research  under  Grant  ®P-AFOSR  77-3281®. 

The  principal  investigators  were  Professor  Michael  Athans  and 
Professor  San joy  Mitter.  The  contract  monitors  were  Major  C.L.  Nefzger, 
Dr.  J.  Bram  and  Dr.  B.  Epstein  of  the  AFOSR  Directorate  of  Mathematical 
and  Information  Sciences. 

Research  was  carried  out  on  the  following  main  topics : 

1.  Realization  Theory  for  Stochastic  Systems 

2.  Linear  and  Non-linear  Filtering 

3 .  Stochastic  Control 

-4.  Adaptive  Control 

Technical  details  of  the  research  may  be  found  in  the  reports,  theses, 
and  papers  cited  in  the  references.  A  list  of  publications  supported 
wholly  or  partially  by  this  grant  is  included  at  the  end  of  this 


report . 


0.  Introduction. 


In  this  report  we  describe  the  work  on  stochastic  and  adaptive 
systems  which  has  been  done  during  the  period  February  1,  1977  to 
January  31,  1982  supported  by  the  grant.  In  the  initial  stages 
of  this  grant,  Professors  T.L.  Johnson  and  A.S.  Willsky  also 
participated  in  this  research  program.  Professor  Johnson's  work 
was  concerned  with  control  of  stochastic  systems  with  finite-state 
compensators.  This  work  was  subsequently  continued  by  Professor 
T.L.  Johnson  under  AFOSR  Grant  AFOSR/F49620-80-C-0002  and  we  refer 
the  reader  to  the  final  report  of  this  last-named  grant  for  details 
of  this  work.  Professor  Willsky 's  work  on  random  fields  was 
initially  partially  supported  by  this  grant.  We  do  not  give 
details  of  this  work  in  this  report  but  refer  the  reader  to  the 
publications  cited  at  the  end  of  this  report. 

The  main  work  performed  under  this  grant  may  be  categorized  as 
follows: 

1)  Stochastic  Realization  Theory 

2)  Linear  and  Nonlinear  Filtering  Theory 

3)  Stochastic  Control 

4)  Adaptive  Control. 

This  research  was  carried  out  by  Professors  M.  Athans  and 
S.K.  Mitter  and  Dr.  L.  Valavani  assisted  by  several  graduate 
students.  Professors  M.H.A.  Davis  of  Imperial  College,  London 
and  Professor  John  Baras  of  University  of  Maryland  also  partici¬ 
pated  in  this  research  as  visiting  scientists  with  support 
extended  by  this  grant. 
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1.  Contributions  Made  in  this  Research  Program. 


We  feel  that  the  major  contributions  of  this  work  have  been  the 
following: 

1)  Proof  of  the  Innovations  conjecture  of  Kailath  (originally  made 
in  1967),  a  major  open  problem  in  non-linear  filtering  since  1967. 

2)  Development  of  a  new  theory  of  non-linear  filtering  based  on  the 
Zakai  equation  leading  to  new  insights  into  existence  and  non-exis¬ 
tence  of  finite-dimensional  filters  and  approximation  methods. 

3)  The  demonstration  of  the  close  connection  between  mathematical 
problems  of  non-linear  filtering  and  quantum  physics. 

4)  Multiple  Integral  Expansions  for  non-linear  filtering  and 
equations  for  best  quadratic  filters. 

5)  Relationship  between  non-linear  filtering  and  stochastic  control  — 
again,  a  major  open  problem  since  the  mid-sixties. 

6)  Initiation  of  a  theory  of  non-linear  stochastic  realization. 

7)  Insights  into  Robust  Stochastic  Control. 

8)  Pinpointing  the  deficiencies  of  existing  adaptive  control  algorithms 
and  a  new  approach  to  the  transient  analysis  of  adaptive  control 


schemes . 


.  Framework  and  Motivation  of  this  Research. 


Our  research  has  been  concerned  with  .fundamental  aspects  of  con¬ 
trolling  linear  and  non-linear  stochastic  systems  in  the  presence  of 
measurement  and  parameter  uncertainties.  In  case  the  uncertainties 
reside  in  the  state  description  of  the  physical  system  and  measure¬ 
ments,  then  we  refer  to  the  control  problem  as  a  stochastic  control 
problem.  If,  in  addition,  there  are  parameter  uncertainties,  then 
the  problem  is  referred  to  as  an  adaptive  control  problem.  This  is 
because,  in  addition  to  state  estimation,  some  form  of  parameter 
identification  scheme  will  be  needed  and  almost  always  the  control 
and  estimation- identification  functions  will  interact  in  a  non¬ 
trivial  nonlinear  and  time-varying  way. 

A  subproblem  of  the  stochastic  and  adaptive  control  problem  is 
the  state  estimation  problem.  Suppose  for  a  moment  we  make  the  as¬ 
sumption'  that  there  are  no  parameter  uncertainties  present  in  the 
dynamical  description  of  the  state  of  the  system.  Then  all  the  pro¬ 
babilistic  information  that  one  can  extract  about  the  "state"  of  the 
system  on  the  basis  of  noisy  measurements  of  the  state  is  contained 
in  the  conditional  probability  density  of  the  state  given  the  obser¬ 
vations.  Indeed,  this  is  the  probabilistic  state  of  the  joint 
physical-measurement  system.  The  recursive  computation  of  the 
probabilistic  state  is  the  state  estimation  problem.  If  this  could 
be  done,  then  one  could  look  for  the  best  controller  as  a  function 
of  this  probabilistic  state,  best  being  judged  in  terms  of  a  suitable 
performance  criterion. 


We  believe  that  wo  have  only  scratched  the  surface  in  understanding 
some  of  these  fundamental  issues  in  modelling,  estimation,  and  control 
of  linear  systems  with  multiplicative  uncertain  parameters  and,  more 
generally,  non-linear  systems.  Our  proposed  research  deals  with  those 
issues  that  we  feel  are  both  fundamental  and  relevent  from  an  engineering 
point  of  view. 

We  would  like  to  mention  that  many  of  the  theoretical  questions  we 
are  posing  have  been  motivated  by  problems  which  are  of  importance  to 
the  U.S.  Air  Force.  Modelling,  estimation,  and  identification  is 
clearly  fundamental  for  application  of  control  theory,  especially  the 
theory  developed  in  the  last  fifteen  years  or  so.  Kalman  filtering 
has  found  widespread  application  in  the  guidance  and  control  of  aero¬ 
space  vehicles,  laser  tracking  and  pointing  systems  and  missile  designs. 
However,  very  few  applications  of  non-linear  filtering  are  known.  The 
extended  Kalman  filter  which  is  often  used  in  practice  is  basically  not 
understood  from  a  scientific  point  of  view.  The  success  of  passive 
tracking  algorithms  requires  advances  in  filtering. 

The  control  of  future  aircraft,  possibly  statistically  unstable, 
is  a  problem  of  continuing  importance  from  the  point  of  view  of 
designing  adaptive  stability  augmentation  systems  that  operate 
reliably  over  a  wide  operating  envelope.  Similarly,  the  control 
of  advanced  jet  engines,  whose  dynamical  characteristics  change 
rapidly  with  operating  conditions,  poses  a  challenging  problem  if 
one  wishes  to  design  a  control  system  which  accommodates  commanded 
thrust  level  changes  rapidly,  while  maintaining  fan  and  compressor 


stability  margins,  and  minimizing  excess  temperature  durations. 
Furthermore,  adaptive  control  appears  particularly  attractive  to 
handle  less  predictable  things  such  as  changes  in  atmospheric  gust 
(particularly  wind  shears  on  final  approach)  and  changes  to  aircraft 
weight  and  mass  center  (e.g. ,  due  to  unusual  loading  distribution  or 
dropping  of  external  stores). 

The  use  of  optical  sensors  and  laser  pointing  and  tracking  systems 
leads  to  new  problems  in  both  stochastic  estimation  and  control.  For 
example,  in  a  laser  tracking  system  the  "glint"  phenomenon  corresponds 
to  noise  in  the  measurement  equation  that  is  multiplicative  rather 
than  additive  in  nature.  Any  stochastic  interception  problem  invol¬ 
ving  a  maneuvering  target  and  with  "angle  only"  sensors  corresponds 
to  a  very  complex  nonlinear  estimation  and  control  problem. 
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3-  Description  of  Research. 


3-  1  NONLINEAR  FILTERING  AND  STOCHASTIC  CONTROL 
3-1*1  Nonlinear  Filtering 

Significant  progress  had  made  in  the  area  of  non-linear  filtering 
by  Professor  Mitter  and  his  graduate  students,  Daniel  Ocone  and 
Larry  Vallot.  This  work  has  been  reported  in  various  papers  and  two 
theses  (References  [1]  to  [10] )  all  supported  by  this  grant.  The 
progress  that  has  been  made  is  on  several  fronts: 

(i)  The  innovations  problem  of  non-linear  filtering  in  the 

form  conjectured  by  Kailath  has  been  finally  settled  [1], 

(ii)  A  new  attack  on  the  non-linear  filtering  problem  based  on 
the  analysis  of  the  Zakai  equation,  proposed  independently 
by  Brockett  and  Mitter  has  had  considerable  success.  In 
fact,  a  major  part  of  the  summer  school  sponsored  by  NATO 
in  Les  Arcs,  France  held  in  July  198C  was  devoted  to  ideas 
proposed  in  [2]. 

(iii)  A  rigorous  formulation  of  the  variational  principle 

underlying  non-linear  filtering  has  been  presented  in 
[3],  This  gives  new  insight  into  questions  of  approxi¬ 
mation  for  non-linear  filters. 

(iv)  It  has  been  shown  that  there  are,  indeed,  deep  connections 
between  non-linear  filtering  and  recent  ideas  of  quantum 
physics  from  a  stochastic  viewpoint  ([2],  [3]). 

(v)  In  [8]  it  has  been  shown  how  these  recent  developments 

can  be  used  to  obtain  sub-optimal  filters  (with  guaranteed 
performance  improvement)  for  bilinear  stochastic  systems. 
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Previous  work  on  non-linear  filtering  has  concentrated  on  the 
equation  for  the  conditional  density  of  the  state  given  the  obser¬ 
vations  over  a  time  interval.  This  equation,  the  so-called  Kushner- 
Stratanovich  equation,  is  a  non-linear  stochastic  partial  differential 
equation  with  an  integral  term.  In  contrast,  the  recent  work  of  Mitter 
has  concentrated  on  the  equation  for  the  unnormalized  conditional 
density,  the  so-called  Duncan-Mortensen- Zakai  equation.  This  is  a 
bilinear  stochastic  partial  differential  equation  which  has  as  its 
input  the  observation  process  and,  hence,  is  considerably  simpler  to 
analyze  than  the  Kushner-Stratanovich  equation.  This  equation  acts 
as  a  universal  recursive  filter,  in  the  sense  that  any  conditional 
statistic  can  be  computed  by  an  integration  with  respect  to  this  un¬ 
normalized  density  and  an  appropriate  normalization. 

A  general  program  for  analyzing  this  equation  and  its  invariance 

properties  was  proposed  in  [2].  In  this  program  a  certain  Lie  algebra 

1  2 

of  operators  with  two  generators  L*  -  —  and  where  L*  is  the 
formal  adjoint  of  the  generator  of  a  diffusion  process  and  L^  is  a 
multiplication  operator  has  an  important  role  to  play.  If  this  Lie 
algebra  is  a  finite  dimensional  than  it  is  an  indication  that  the 
Zakai  equation  has  a  finite-dimensional  statistic.  In  fact,  the  n- 
dimensional  Kalman  filtering  problem  gives  rise  to  a  (2n+2) -dimensional 
Lie  algebra  which  is  an  n-dimensional  generalization  of  the  Oscillator 
algebra  of  some  fame  in  quantum  physics.  Indeed,  the  Kalman  filter 
occupies  the  same  role  as  the  Harmonic  oscillator  or  the  free  field 
in  quantum  physics  ([2],  [3]).  A  by-product  of  this  approach  is  that 
the  Zakai  equation  for  the  Kalman  filter  has  a  finite-dimensional 
statistic  even  if  the  initial  density  is  a  non-gaussian.  Soon  after 
these  ideas  became  known,  Bones  exhibited  finite-dimensional  filters 


for  a  class  of  filtering  problems  with  non-linear  drift  and  it  was  shown 
by  Mitter  [2]  that  the  filtering  problems  of  Benes  were  '‘gauge  equiva¬ 
lent"  to  the  Kalman  filtering.  In  the  doctoral  dissertation  of 
D.  Ocone  [7]  it  is  proved  that  for  scalar  diffusion  processes  defined 
over  the  whole  real  line,  the  Kalman  filtering  problem  and  the  gauge 
equivalent  Benes  filtering  problems  were  the  only  problems  admitting 
finite-dimensional  filters. 

In  [2] ,  it  was  shown  that  the  Lie  algebra  of  operators  corres¬ 
ponding  to  a  large  class  of  filtering  problems  was  infinite-dimensional 
and  simple.  In  particular,  it  was  shown  that  the  Lie  algebra  of 
operators  corresponding  to  the  cubic  sensor  problem  (extensively  investi¬ 
gated  by  R.  S.  Bucy)  was  the  Weyl  algebra  (the  Lie  algebra  of  partial 
differential  operators  of  all  order)  which  is  known  to  be  infinite¬ 
dimensional  and  simple.  This  result  was  obtained  independently  by 
M.  Hazewinkel  and  S.  Marcus.  It  was  conjectured  in  [2]  that  no 
finite-dimensionally  computable  statistic  existed  for  this  problem. 

This  has  now  been  proved  by  H.  Sussman. 

In  proving  the  above  result,  one  constructs  a  proof  of  the 
Ansatz  of  R.W.  Brockett,  namely,  that  if  there  exists  a  finite¬ 
dimensional  filter  than  there  must  exist  a  certain  Lie  Algebra  homo¬ 
morphism  between  the  Lie  Algebra  of  operators  corresponding  to  the 
filtering  problem  and  the  Lie  Algebra  of  certain  vector  fields.  For 
the  cubic  sensor  problem  this  was  done  by  M.  Hazewinkel  and  S.  Marcus. 

To  prove  the  theorem  of  the  non-existence  of  finite-dimensionally 
computable  statistic  one  needs  a  rigorous  proof  of  the  existence  and 
uniqueness  of  solutions  of  the  Zakai  equation.  In  joint  work  of 


Mitter  with  J.S.  Baras  and  D.  Ocone  (preliminary  report  in  [9])  this 
has  been  accomplished  when  the  observation  map  is  a  polynomial. 

In  D.  Ocone's  doctoral  thesis  [7],  the  problem  of  expanding 
the  unnormalized  conditional  density  in  a  Ito-Wiener  Series  has 
been  solved.  Ocone  also  obtains  a  multiplication  formula  for 
multiplying  two  multiple  Ito  integrals  and  subsequently  expanding 
it  in  an  Ito-Wiener  Series.  He  has  also  obtained  integral  equations 
for  the  kernels  corresponding  to  best  polynomial  filters  and  shown 
existence  and  uniqueness  of  solutions. 

In  L.  Vallot's  S.M.  thesis  [8],  a  theoretical  and  experimental 
investigation  of  filtering  for  bilinear  systems  has  been  undertaken. 
As  examples,  he  has  considered  the  phase-lock  loop  problem  and  also 
the  cubic  sensor  problem.  He  has  shown  how  tensor  and  Fock  space 
ideas  expressed  in  [2]  can  be  usefully  utilized  to  obtained  sub¬ 
optima],  filters  which  guarantee  performance  improvement  (in  the 
mean  square  sense) . 

In  summary,  a  framework  has  been  established  for  the  theo¬ 
retical  and  experimental  investigation  of  a  broad  class  of  non¬ 
linear  filtering  problems.  It  is  believed  that  these  ideas  can  be 
used  to  design  practical  non-linear  filters  and  evaluate  their 
performance. 

3.1.2  On  the  Relationship  Between  Non-Linear  Filtering 
and  Stochastic  Control 

It  has  long  been  felt  that  there  is  a  variational  principle  unde 
lying  non-linear  filtering.  Indeed,  in  the  early  sixties,  a  formu¬ 
lation  of  the  non-linear  filtering  problem  as  a  non-linear  least- 
squares  problem  was  proposed  first  by  Bryson  and  Frazier  and, 
subsequently,  by  Mortensen.  These  formulations  were,  however, 


deterministic  formulations  and  had  the  inherent  mathematical  difficulty 
of  working  with  white  noise,  the  formal  differential  of  the  Wiener 
process  (which  is  almost  surely,  not  differentiable).  It  was  also 
known  that  these  formulations  gave  the  correct  answer  for  the  Kalman 
Filtering  problem. 

In  [11]  Professor  Mitter  (joint  work  with  Professor  Fleming  of 
Brown  University)  has  shown  that  associated  with  the  Zakai  equation 
of  non-linear  filtering  is  a  Bellman-Hamilton-Jacobi  equation  which 
corresponds  to  a  perfectly  observable  stochastic  control  problem. 

The  solution  of  one  equation  leads  to  a  solution  of  the  other.  In 
particular,  the  Kalman  filtering  problem  corresponds  to  a  perfectly 
observable  linear  quadratic  gaussian  stochastic  control  problem. 

Indeed,  this  explains  in  the  clearest  possible  way  the  duality 
principle  originally  proposed  by  Kalman.  The  difficulty  of  the 
non-differentiability  of  the  Wiener  paths  is  avoided  by  working  with 
the  robust  form  of  the  non-linear  filtering  equations  as  developed 
by  Clark  and  Davis  (M.H.  Davis  of  Imperial  College  was  a  visitor 
at  M.I.T.  in  1979  and  was  partially  supported  by  this  grant). 

These  ideas  have  important  consequences.  It  opens  the  way  for 
developing  algorithms  for  non-linear  filtering  using  stochastic  control 
ideas.  It,  for  the  first  time,  explains  how  the  extended  Kalman 
filtering  algorithm  really  works.  Finally,  it  has  important  impli¬ 
cations  in  the  development  of  a  theory  of  stochastic  dissipativeness. 

3.1.3  Concluding  Remarks 

The  developments  in  non-linear  filtering  reported  here  have  many 
consequences.  For  example,  parameter  identification  problems  can  be 
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considered  to  be  special  cases  of  non-linear  filtering  problems  and 
the  ideas  discussed  here  will  be  important  in  obtaining  a  better 
understanding  of  identification.  It  is  also  felt  that  the  relation¬ 
ship  between  non-linear  filtering  and  stochastic  control  exhibited 
here  will  play  a  role  in  obtaining  a  better  understanding  of 
stochastic  adaptive  control. 

3.2  STOCHASTIC  REALIZATION  THEORY 

The  problem  of  stochastic  realization  theory  could  be  thought  of 
as  the  construction  of  state-space  representations  for  stochastic  pro¬ 
cesses.  The  problem  is  essentially  solved  in  the  case  where  the  ob¬ 
served  stochastic  process  is  a  stationary  Gaussian  process  [1 2]  .  In 
joint  work  with  A.  Lindquist  and  G.  Picci  5.2]  ,  we  have  initiated  work 
where  the  stationary  process  has  finite-energy  and  has  an  innovation 
representation  in  the  form  of  a  wiener- Ito  series.  We  have  also 
investigated  the  special  case  where  the  observed  process  is  the  out¬ 
put  of  a  bilinear  stochastic  differential  equation.  We  feel  that  this 
work  has  important  ramifications  for  non-linear  filtering  theory. 

3.3  ADAPTIVE  CONTROL 

For  the  past  several  years  an  intensive  study  of  characteristics 
of  existing  direct  adaptive  control  algorithms  has  been  conducted  by 
Drs.  Athans,  and  Valavani  with  the  assistance  of  Drs.  Stein  and  Sastry 
and  several  students.  The  initial  emphasis  was  to  understand  the 
transient  behavior  of  existing  direct  adaptive  control  algorithms  and 


their  robustness  to  unmodelled  dynamics  and  observation  noise. 


The  first  phase  of  this  research  was  devoted  to  digital  simulation 
studies  and  a  brief  paper  [131described  the  simulation  results.  A 
major  part  of  the  research  that  was  carried  out  during  this  phase  was 
reported  in  the  annual  report  prepared  for  the  AFOSR  for  the  previous 
year. 

From  the  simulation  results,  it  became  self-evident  that  no 
consistent  pattern  with  respect  to  the  adaptation  process  could  be 
predicted.  Nonetheless,  the  simulation  results  confirmed  our  sus¬ 
picions  that  the  class  of  adaptive  algorithms  considered  were  char¬ 
acterized  by: 

(a)  high-frequency  control  signals  characteristic  of  a 
high-bandwidth  system 

(b)  the  extreme  sensitivity  of  the  algorithm  to  un- 
mcdelled  high-frequency  dynamics  which  can  result 
in  unstable  closed  loop  behavior 

(c)  lack  of  robustness  to  observation  noise. 

Motivated  by  the  simulation  results  in  [13]  a  decision  was  made  to 
initiate  an  analytical  investigation  into  the  nature  and  properties  of 
several  available  direct  control  algorithms.  The  focus  of  the 
analytical  effort  was  to  understand: 

(a)  the  dependence  of  the  closed  loop  adaptive  system 
bandwidth  upon  the  amplitude  and  frequency  content 
of  the  reference  input  signal 

(b)  the  robustness  of  the  adaptive  control  system  to  un¬ 
modelled  high  frequency  dynamics 


<c) 


the  impact  of  sensor  noise. 


To  gain  a  basic  understanding  it  was  assumed  that  the  controlled  plant 
was  a  simple  first-order  system;  the  rationale  was  that  if  undesirable 


performance  and  robustness  characteristics  were  encountered  for  first- 
order  systems,  one  could  certainly  conjecture  that  the  same  problems 
would  arise  in  more  interesting  high  order  systems. 

i.  3' .  1  An  Analytical  Methodology  in  the  Study  of  Convergence 
Patterns  of  Adaptive  Algorithms. 

A  recent  paper  [14]  summarizes  our  progress  to  date.  We  have 
been  successful  in  devising  an  analytical  technique,  based  upon 
linearization,  which  ve  call  the  final  approach  analysis,  that  can 
be  used  to  analyze  the  dynamic  properties  of  several  available  direct 
adaptive  control  algorithms,  both  in  the  continuous-time  case  and 
the  discrete- time  case.  In  particular,  this  method  can  be  used  to 
predict  the  behavior  of  the  adaptive  systems  with  respect  to  para¬ 
meter  convergence,  sensitivity  to  unmocelled  dynamics,  and  impact 
of  observation  noise. 

As  explained  in  more  detail  in  [14]  the  final  approach  analysis 
method  is  valid  during  the  final  stage  of  adaptation  in  which  the 
output  error  is  small.  During  this  phase  one  can  linearize  the 
general  nonlinear  time-varying  differential  (or  difference,)  equations 
linking  the  dynamics  of  the  output  error  to  those  of.  the  parameter 
adjustment  algorithm.  One  then  obtains  a  set  of  linear  differential 
or  difference  equations  which  are  either  time-varying  or  time-invariant 
depending  upon  the  nature  of  the  reference  (command)  inputs  and  out¬ 
puts.  It  then  becomes  possible  to  analyze  the  behavior  of  the  linearized 
dynamics  using  available  results  in  linear  systems  theory.  When  the 
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resultant  dynamics  are  time-invariant,  even  simple  root-locus  type 
of  plots  can  be  used  to  predict  the  asymptotic  perforamance  of  the 


adaptive  system  with  respect  to  oscillatory  behavior  and  possible 
instability  in  the  presence  of  unmodelled  dynamics. 

The  technical  results  in  [14] demonstrate  the  value  of  the 
final  approach  analysis;  it  has  been  used  to  analyze  the  behavior 
of  the  adaptive  systems  when  the  algorithms  of  Narendra  and 
Valavani  [15]  Feuer  and  Morse  [1.6]  Narendra,  Lin,  Valavani  [17] 

Morse  [18]  Narendra,  Lin  [19]  Landau  and  Silveira  (20]  [21]  and 
Goodwin,  Ramadge  and  Caines  [22]  were  employed.  For  the  base-line 
first-order  example  considered,  all  algorithms  considered  were 
found  (in  different  degrees)  to  suffer  from  the  viewpoint  of 
yielding  high-bandwidth  closed-loop  systems  which  can  excite  high- 
frequency  dynamics. 

It  is  important  to  place  the  method  of  final  approach  analysis 
into  perspective  regarding  both  completed  results  as  well  as  relevant 
current  research  directions  which  include: 

(1)  High-order  Systems  -  It  is  possible  to  extend  the  final 
approach  analysis  method  to  high  order  systems,  using  either 
output  feedback  or  state  feedback,  and  examine  analytically 
the  coupled  dynamic  characteristics  of  the  output  error  and 
parameter  adjustment  logic,  through  the  study  of  the  resultant 
linear  time-varying  differential  or  difference  equations. 

This  has  been  already  done  (but  not  formally  documented,  as 
yet)  for  several  of  the  algorithms  already  considered  in  [14] 
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(2)  Effects  of  Input  and  Output  -  The  linear  time-varying 
dynamical  equations  that  arise  from  the  final  approach  analysis 
have  a  very  special  structure.  By  this,  we  mean  that  the 

analysis  leads  to  vector  differential  equations  of  the  form 

• 

!Z(t)  =  F(t)  52 (t) ;  however,  the  matrix  F(t)  has  a  very  special 
form  that  allows  one  to  study  its  characteristic  polynomial 
in  a  relatively  straight-forward  manner.  In  particular, 

(as  demonstrated  in'the  specific  examples  analyzed  in  [14],) 
the  command  reference  input  r(t)  and  output  y(t)  appear  non- 
linearly  in  certain  elements  of  the  matrix  F(t) ;  study  of 
the  eigenvalues  of  F(t)  reveals  that  both  the  magnitude 
and  the  frequency  of  the  input  and  output  signals  impact 
the  bandwidth  of  the  adaptive  closed-loop  system  and  hence 
its  robustness  to  high-frequency  ivnamics. 

(3)  Constructive  Use  of  the  Final  Approach  Analysis  -  The  linear 
dynamic  equations  that  arise  through  the  use  of  the  final  approach 
analysis  in  conjunction  with  any  adaptive  algorithm  can  be  used 

in  a  constructive  way  to  modify  the  algorithm  so  as  to  alleviate 
to  the  extent  possible  its  practical  shortcomings.  As  a  specific 
example,  the  results  in  Section  4.3  of  [14]  demonstrate  how  to 
adjust  the  control  gains,  as  a  nonlinear  function  of  the  input 
and  output  so  as  to  improve  the  performance  of  the  tfarendra,  Lin, 
Valavani  [17]  and  Morse  [18] algorithms  in  the  presence  of  unmodclled 
dynamics.  Similarly,  the  results  in  Section  5.2  of  [14]  demonstrate 
how  to  properly  select  the  gains  in  the  Landau-Silveira  [20] ,  [21] 
algorithms. 
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The  main  motivation  for  the  modifications  introduced  in  the 


above  can  essentially  be  attributed  to  the  need  to  limit  the  band¬ 
width  of  the  adaptive  system  so  as  not  to  excite  the  high  fre¬ 
quency  unmodelled  dynamics  which  invariably  drive  it  to  instability. 

In  the  specific  cases  cited  above,  this  was  achieved  by  app  o- 
priately  controlling  the  resulting  final  approach  root  locus  pattern, 
either  by  keeping  the  root  locus  parameter  (gain)  within  bounds,  as 
a  function  of  the  reference  input  and  process  output,  or  by  intro¬ 
ducing  an  additional  zero  at  an  appropriate  location,  so  as  to  reduce 
the  order  of  the  root-locus  pattern,  which  is  already  enhanced  by 
the  presence  of  the  unmodelled  poles.  Unfortunately,  these  improve¬ 
ments  cannot  be  generalized  to  deal  with  the  truly  transient  phase 
without  seriously  affecting  the  stability  arguments,  or  to  handle 
any  number  of  unmodelled  poles  beyond  a  certain  frequency.  Clearly, 
a  more  substantial  remedy  is  necessary.  We  defer  a  discussion  on 
this  to  the  subsequent  subsection.  Also,  a  discussion  of  the  effects 
of  observation  noise  and  disturbances  on  the  performance  of  adaptive 
algorithms  is  included  in  a  subsection  by  itself. 

Justifiably,  since  the  final  approach  analysis  is  based  upon 
dynamic  linearization  under  the  assumption  that  the  output  error  is 
small,  it  cannot  predict  the  dynamic  behavior  of  the  adaptive  system 
during  its  transient  (start-up)  phase.  The  simulation  results  of  [13] 
suggest  that  even  more  complex  dynamic  effects  are  present.  Thus, 
the  final  approach  analysis  must  be  viewed  as  a  necessary,  but  by  no 
means  sufficient,  step  in  the  analysis  and  design  of  adaptive  algo¬ 


rithms. 


In  spite  of  its  limitations,  we  believe  that  the  final  approach 
analysis  is  a  useful  tool,  since  it  can  predict  undesirable  characte¬ 
ristics  of  wide  classes  of  adaptive  algorithms  in  the  final  phase  of 
the  adaption  process;  these  characteristics  are  apt  to  persist  (or 
get  even  worse)  in  the  transient  start-up  phase.  Moreover,  the  final 
approach  analysis  can  suggest  ways  of  modulating  the  control  gains, 
in  a  nonlinear  manner,  to  improve  performance  in  the  absence  of  high 
frequency  modelling  errors.  At  this  stage  of  understanding  the  resul¬ 
tant  transient  start-up  characteristics  can  be  evaluated  only  by 
simulation;  current  research  is  focusing  on  analytical  results  for 
this  aspect  of  the  adaptive  problem. 

3.3.2  Model  Assumptions  and  Fundamental  Questions 
From  the  foregoing  discussion,  the  inevitable  conclusion  is  that 
a  good  (practical)  adaptive  control  loop  must  adjust  its  bandwidth 
(cross-over  frequency)  in  such  a  manner  so  that  it  does  not  excite 
unmodelled  high  frequency  dynamics.  To  put  it  another  way,  the 
adaptive  loop  must  remain  stable  in  the  presence  of  -unstructured 
modelling  uncertainty  which  always  exists  and  cannot  be  adequately 
modelled  in  any  physical  system.  On  the  other  hand,  the  adaptive 
control  system  must  also  be  able  to  provide  performance  improvement 
in  the  case  of  plant  structured  uncertainty,  typically  exhibited 
when  the  parameters  in  the  differential  equations  that  are  used  to 
model  the  plant  in  the  low  frequency  region  vary  within  a  bounded 
set.  The  adaptive  system  must  exhibit  good  command  -  following  and 
disturbance  -  rejection  properties  in  the  low  frequency  region  whore 
the  structured  model  uncertainty  predominates. 
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Our  research  experience  to  date  points  to  the  existence  of  a 
fundamental  conflict  in  many  adaptive  control  schemes.  To  compen¬ 
sate  for  structured  uncertainty  and  performance  the  adaptive  scheme 


may  wish  to  increase  the  cross-over  frequency.  On  the  other  hand, 
the  presence  of  unstructured  uncertainty  places  an  upper  bound  on 
the  cross-over  frequency  in  order  to  maintain  stability.  Good 
practical  adaptive  algorithms  must  be  "smart"  enough  to  recognize 
this  fundamental  conflict,  and  adjust  their  cross-over  frequency. 

The  mathematical  assumptions  that  have  led  to  all  available 
adaptive  control  algorithms  have  taken  into  account  the  existence  of 
structured  uncertainty  but  they  have  neglected  completely  the  issue 
of  unstructured  uncertainty.  To  avoid  such  an  undesirable  behavior 
as  discussed  in  the  preceding,  the  mathematical  assumptions  must 
include  information  or.  the  unstructured  uncertainty.  The  question 
is:  what  level  of  detail  can  one  provide  to  the  mathematics  about 
the  unstructured  uncertainty.. 

As  a  natural  consequence  of  the  above  considerations,  we  have 
adopted  currently  a  fundamentally  different  structure  of  the  model 
for  the  plant  to  be  controlled,  described  below: 

3.3.3  Frequency  Domain  Description 

In  the  single-input-single-output  (SISO)  case,  we  assume  that 
the  unknown  plant  has  the  transfer  function,  from  the  scalar  control 
u(t)  to  the  scalar  output  '/ (t) ,  given  by: 
g(s)  =  go(s,0)  [1  +  £(s)] 
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The  parameter  vector  0  in  the  transfer  function  g(s,0)  gives 
rise  to  the  structured  modelling  uncertainty,  and  we  assume  that  it  is 
a  good  model  for  the  plant  at  low  frequencies.  The  parameter  0  is 
assumed  to  be  in  a  closed  bounded  set}  the  relative  degree  of  gQ(s,0) 
will  not  change  as  the  parameter  vector  0  changes*.  In  fact,  a  nominal 
value  of  0  may  even  be  known  which  can  be  used  in  the  adaptive  control 
algorithm  before  any  real  time  measurements  (initialization  problem) . 

The  transfer  function  Z(s)  gives  rise  to  the  unstructured  modelling 
uncertainty  (high  frequency  bending  modes,  small  time-delays,  non¬ 
minimum  phase  zeros  at  high  frequencies,  etc.).  Jl(s)  represents  a 

multiplicative  perturbation  to  g  (s,0)  under  the  present  formulation. 

o  ~ 

An  analogous  formulation  with  parallel  definitions  as  in  the  SISO 
can  be  applied  to  the  MIMO  problem  as  follows: 

(a)  For  unstructured  modelling  uncertainty  reflected  at 
the  plant  input 

G(s)  =  G  0  (s,0)  [I  +  L(s)  ] 

(b)  For  unstructured  modelling  uncertainty  reflected  at  the 
plant  output 

G(s)  =  [I  +  L(s)]G6(s,0) . 


*In  aerospace  applications,  the  parameter  0_  models  the  effect  of 
changing  dynamic  pressure  and  geometry  upon  the  aerodynamic  co¬ 
efficients  in  the  equations  of  motion  (typically  rigid  translational 
and  rotational  dynamics ) . 
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Our  approach  takes  the  point  of  view  that  we  know  very  little  about 
the  unmodelled  high  frequency  dynamics.  At  sufficiently  high  frequencies, 
S.(s),  and  the  elements  of  L{s) ,  will  be  characterized  by  +180°  phase  un¬ 
certainty.  We  also  do  not  know  very  much  about  the  order  of  such  high- 
frequency  dynamics . 

We  claim  that  the  only  reasonable  information  that  a  control 
engineer  has  about  high  frequency  errors  is  as  follows: 

(a)  They  are  negligible  at  low  frequencies 

(b)  There  is  a  frequency,  to^,  in  which  the  magnitude  of 
the  unstructured  modelling  uncertainty  becomes 
significant 

(c)  They  are  dominant  at  high  frequencies. 

The  way  we  model  the  existence  of  Ms)  or  L(s)  in  the  adaptive 
control  problem  is  to  assume  than  there  exists  a  scalar  function  of 
frequency,  m(to)  such  that 

|£(jto)|<  m(to),  Va  for  the  SISO  case, 
and  0  (L(jto))<m  (to),  V  to  for  the  M1M0  case, 

ITtc&X 

where  0  denotes  the  maximum  singular  value  of  L(jto). 
max  — 

The  function  m(to)  is  small  at  low  frequencies  and  is  a  monotonically 

increasing  function  of  to.  The  frequency  to  at  which  m(to  )  =  1,  will 

u  u 

be  referred  to  as  the  uncertainty  cross-over  frequency. 

The  existence  of  the  unstructured  uncertainty  places  an  upper 
bound  on  the  bandwidth  of  the  closed-loop  system,  whether  the  closed- 
loop  system  is  adaptive  or  not.  If  one  assumes  that  the  "best"  non- 
adaptive  controller  has  a  maximum  gain  crossover  frequency  for  all  0 
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in  the  allowable  parameter  set  of  &)*,  then  for  guaranteed  stability  in 

the  presence  of  unstructured  uncertainty  we  must  have  co*<go  .  On  the 

c  u 

other  hand,  use  of  an  adaptive  controller  is  justified  if  the  perfor¬ 
mance  of  the  "best"  nonadaptive  controller  is  not  good  enough  for  a 
specific  problem.  In  many  applications,  whenever  the  parameter 
vector  0  is  not  at  its  worst  value,  an  adaptive  algorithm,  after  the 
transient  adaptation  phase,  should  converge  to  a  compensator  with  cross¬ 
over  frequency  that  satisfies 

★ 

<  0)  .  <  6JU 

c  cZ 

for  performance  and  robustness.  Unfortunately,  our  investigations  have 
shown  that  none  of  the  currently  existing  algorithms  satisfy  the  above 
criterion. 

The  problem  as  we  see  it,  arises  with  the  nature  of  the  information 
available  to  the  explicit  or  implicit  adaptive  controller  which  relies 
for  its  parameter  adjustments,  on  some  form  of  output  signal  (s)  cor¬ 
relation.  This,  incidentally,  is  true  of  every  adaptive  algorithm  pro¬ 
posed  in  the  literature  (dual  control,  self-tuning  regulators,  model- 
reference  adaptive  control).  When  the  requirement  co  is  "temporarily" 

violated  (due  to  the  initial  "hunting"  behavior  of  adaptive  algorithms), 
the  closed  loop  system  becomes  temporarily  unstable,  its  high  frequency 
dynamics  in  Ms)  (or  L(s))  get  excited,  and  hence  the  measured  outputs 
contain  signals  (the  models  of  Ms)  or  L(s))  for  which  by  assumption 
we  know  nothing  about,  in  view  of  the  magnitude  and  phase  uncertainty  for 
The  output  correlation  algorithm,  consequently,  must  be  able  to 
quickly  separate  the  relevant  information  in  the  output  signals  from 
those  that  the  algorithm  knows  nothing  about,  and  at  the  same  time 


carry  out  rapidly  the  necessary  (explicit  or  implicit)  change  in  the 


controller  parameters  in  the  correct  direction!  The  results  in  [14] 
demonstrate  that  many  of  the  existing  adaptive  control  algorithms 
cannot  handle  this  phenomenon,  and  the  closed  loop  system  breaks 
into  total  instability. 

The  focal  point  of  our  current  research  is  how  to  characterize 
the  impact  of  the  unstructured  uncertainty  upon  the  output  measure¬ 
ments  and  to  consequently  reflect  this  in  the  parameter  adjustment 
mechanism.  Both  time-domain  as  well  as  frequency  domain  methods 
are  being  employed  in  these  investigations,  the  baseline  for  which 
is  being  provided  by  the  final  approach  analysis,  and  our  results 
will  be  reported  at  a  future  date,  upon  completion. 

3.3.4  Disturbances  and  Sensor  Measurement  Noise 

Although  the  MRAC  algorithms  are  explicitly  designed  for  good 
command- following  (the  cheer'/  neglected  to  date  disturbance  rejection) 
the  specific  mechanism  employed  for  achieving  the  necessary  high  loop 
gain, for  this  purpose,  is  unsatisfactory  from  a  practical  point  of 
view.  All  MRAC  type  algorithms  that  we  are  aware  of  do_  not  follow 
the  time-honored  rule  of  augmenting  the  plant  dynamics  with  inte¬ 
grators  and  other  disturbance  rejecting  transfer  functions  which 
naturally  make  the  loop  transfer  function  have  a  large  magnitude  in 
the  command- following/disturbance-rejection  frequency  range.  We  have 
found  that  standard  MRAC  algorithms  increase  the  forward  loop  ampli¬ 
fication  at  all  frequencies,  thus  forcing  the  closed-loop  system  to 
have  a  wide  bandwidth,  which  further  exasperates  the  problem  of 
exciting  the  high-frequency  dynamics.  Our  current  work  here  primarily 
relies  on  extending  the  robustified  LQG  approach  to  suitably  modify 
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existing  MRAC  and  self-tuning  regulator  algorithms  so  that  they  can 
handle  both  known  and  unknown  dynamics  in  the  definition  of  the  struc¬ 
tured  uncertainty. 

Sensor  measurement  noise,  although  it  initially  enters  additively, 
it  readily  becomes  multiplicative  by  entering  the  closed  loop  system 
through  the  parameter  adjustment  law. 

The  final  approach  analysis  can  also  be  employed  in  this  case  to 
analyze  the  effects  that  the  addition  of  the  observation  noise  has 
on  the  adaptive  algorithm.  Section  6  of  [14] illustrates  how  the  metho¬ 
dology  can  be  used  to  analyze  a  particular  algorithm,  and  how  to  modify 
the  algorithm  so  that  parameter  errors  are  not  increased  thus  leading 
to  a  wide  bandwidth  system. 

More  specifically,  the  analysis  in  [14]  shows  that  the  noise  n(t) 

enters  the  (linearized)  system  both  linearly  as  well  as  in  a  quadratic 
2  2 

form  n~  (t) .  The  effect  of  the  term  n  ft)  is  always  to  increase  the 
parameter  errors  which  correspond  to  the  difference  of  the  actual 
closed-loop  pole(s)  from  the  model  reference  pcle(s).  Hence,  the  band¬ 
width  of  the  closed- loop  system  increases  and,  although  it  remains  stable 
for  exact  modelling  [13]  it  is  clear  that  if  the  presence  of  sensor  noise 
causes  the  adaptive  loop  bandwidth  to  increase,  eventually  the  high  fre¬ 
quency  dynamics  will  get  excited  and  instability  will  result. 

We  emphasize  here  that  the  increase  in  bandwidth  rather  than 
stability  (under  assumptions  of  proper  modelling)  is  the  crucial  issue. 

For  the  discrete-time  systems,  in  particular,  mean-square  boundedness 
proofs  for  the  output  errors  have  been  obtained  [23]  under  the  assumptions 


of  proper  modelling  and  positive  realness  of  the  noise  transfer 
function.  Earlier,  Ljung  {24],  obtained  local  convergence  results 
for  stochastic  approximation  type  adaptive  algorithms  also,  under 
the  assumptions  of  stationarity  of  the  closed- loop  signals  and 
positive  reality  of  the  error  transfer  function.  Section  7  of  [ 14] 
yet  contains  what  we  consider  to  be  the  first  proof  in  the  literature 
to  yield  mean-square  boundedness  of  the  parameter  errors  as  well. 

The  problem,  therefore,  with  sensor  noise  is  again  to  charac¬ 
terize  its  effect  on  the  output (s)  and  to  appropriately  compensate 
the  adaptive  laws  for  this.  Some  discussion  for  final  approach 
improvement  is  contained  in  [14]  ,  although  the  problem  in  its 
generality  has  to  be  treated  using  the  same  methodology  discussed 
in  the  preceding  subsection,  in  the  context  of  unmodelled  dynamics. 
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